On the Bayesian “occam Factors” Argument for Occam’s Razor
نویسنده
چکیده
This paper discusses some of the problematic aspects of the Bayesian first-principles “proof” of Occam’s razor which involves Occam factors. Although it is true that the posterior for a model is reduced due to Occam factors if that model is capable of expressing many functions, the phenomenon need not have anything to do with Occam’s razor. This paper shows this by i) performing reductio ad absurdum on the argument that the Occam factors effect implies Occam’s razor; ii) presenting an alternative Bayesian approach which explicitly does not result in Occam’s razor; and finally iii) disentangling the underlying problem with viewing the Occam factors argument as a proof or “automatic embodiment” of Occam’s razor.
منابع مشابه
Solomonoff Prediction and Occam’s Razor
Algorithmic information theory gives an idealized notion of compressibility, that is often presented as an objective measure of simplicity. It is suggested at times that Solomonoff prediction, or algorithmic information theory in a predictive setting, can deliver an argument to justify Occam’s razor. This paper explicates the relevant argument, and, by converting it into a Bayesian framework, r...
متن کاملOccam's Razor
The Bayesian paradigm apparently only sometimes gives rise to Occam’s Razor; at other times very large models perform well. We give simple examples of both kinds of behaviour. The two views are reconciled when measuring complexity of functions, rather than of the machinery used to implement them. We analyze the complexity of functions for some linear in the parameter models that are equivalent ...
متن کاملHow Occam's Razor Provides a Neat Definition of Direct Causation
In this paper we show that the application of Occam’s razor to the theory of causal Bayes nets gives us a neat definition of direct causation. In particular we show that Occam’s razor implies Woodward’s (2003) definition of direct causation, provided suitable intervention variables exist and the causal Markov condition (CMC) is satisfied. We also show how Occam’s razor can account for direct ca...
متن کاملOccam's Razor and a Non-Syntactic Measure of Decision Tree Complexity
Occam's razor, attributed to the fourteenth century English philosopher William of Occam, states: “plurality should not be assumed without necessity.” The machine learning interpretation of Occam’s razor is that if two models have the same performance on the training set, choose the simpler. Decision tree learning widely uses Occam’s razor. Popular decision tree generating algorithms are based ...
متن کاملA Syntactic Justification for Occam’s razor
Informally, Occam’s razor states, “Given two hypotheses which equally agree with the observed data, choose the simpler”, and has become a central guiding heuristic in the empirical sciences and in particular machine learning. We criticize previous arguments for the validity of Occam’s razor. The nature of hypotheses spaces is explored and we observe a correlation between the complexity of a con...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1992